{/* banner prompt: A playful, polished 3D clay-render illustration in a soft matte palette of sage green, warm cream and terracotta orange. The scene shows a chunky, rounded, stylized NVIDIA graphics card sculpted from smooth modelling clay, sitting on a flat cream surface. From one end of the card a long ribbon of pale paper unfurls and coils gently into a neat spiral stack, like a scroll of memory. A tiny terracotta pot holding a small green bonsai tree with rounded clay leaves rests on top of the card. Everything has a tactile hand-sculpted matte finish with gently rounded edges, soft diffuse studio lighting, warm subtle shadows, shallow depth of field, and a simple uncluttered pastel background. Wide 16:9 composition, centred, clean and minimal. */}

Last month I put a [27-billion-parameter reasoning model on this same RTX 3060](/blog/2026/08/18-now-it-thinks-a-27b-language-model-on-the-same-rtx-3060/) — Qwen3.8-27B, at 4-bit, about 18GB. It worked, but only just: a bit more than half the model sat on the card and the rest streamed from system RAM, and it typed back at roughly reading speed, about five words a second.

This post is the same model — and a much better story. Someone re-compressed Qwen3.8-27B down to **under 6GB**, small enough that it now fits *entirely* on the 12GB card with room to spare. It runs about **four times faster**. And with the space freed up, it can hold something the 4-bit version never could: a genuinely enormous context — up to **262,000 tokens**, a fat novel's worth of text, read all at once.

## The machine (still the same one)

Same box as every post in this series: an RTX 3060 with **12GB of VRAM**, 64GB of system RAM, Debian. Nothing upgraded. The card is the constant; only what we ask of it changes.

## What changed: ternary instead of 4-bit

Last time, quantization squeezed each of the model's numbers down to 4 bits. This model goes further — it uses **ternary** quantization, where almost every weight is stored as just one of three values: -1, 0, or +1. That is a hair under two bits per weight, and it drops the file from over 50GB in full precision to **5.95GB**.

The payoff is that the model no longer has to be split between the card and RAM. All of it lives on the GPU, where the math is fast, and none of it has to be streamed across from system memory. That single change — everything on-card instead of 57% on-card — is most of where the speed-up comes from.

## The honest numbers

Same model, same card, last month versus this month:

<Table data={
    {
        headers: ['', 'Last month (4-bit)', 'This month (ternary)'],
        rows: [
            ['Size on disk', '~18 GB', '5.95 GB'],
            ['Runs on GPU', '~57% on card, rest from RAM', '100% on card'],
            ['VRAM used', '11 of 12 GB', '~8 GB (at a 32K context)'],
            ['Speed', '~4.7 words/sec', '~20 words/sec (about 27 tokens/sec)'],
            ['Math, coding, reasoning', 'Correct', 'Correct'],
        ]
    }
} />

Twenty words a second is a comfortable reading pace that no longer feels like waiting. For a single thoughtful answer on your own machine, it is genuinely pleasant to use.

## The part you actually asked about: how much can it remember?

"Context" is how much text a model can hold in its head at once — the question you ask, plus every document, transcript, or code file you paste in with it. The bigger the context, the more it can read before it starts forgetting the start.

Once the weights only take ~6GB, the rest of the 12GB card is free for exactly this. So I measured it: how large a context actually fits before the card runs out of memory. The lever is the *KV cache* — the running memory the model keeps of everything it has read so far — and you can store that cache at different precisions, just like the weights.

<Table data={
    {
        headers: ['KV cache precision', 'Largest context that fits', 'VRAM used', 'Speed'],
        rows: [
            ['16-bit (default)', '~64K tokens (about 48,000 words)', '~10.3 GB', '~27 tokens/sec'],
            ['8-bit', '128K tokens (about 96,000 words)', '~11.1 GB', '~26 tokens/sec'],
            ['4-bit', '262K tokens (about 196,000 words)', '~12 GB (tight)', '~26 tokens/sec'],
        ]
    }
} />

So the answer is: on a 12GB card with the GPU to itself, you comfortably get a **64K-token** context out of the box, **128K** by storing the cache at 8-bit, and the model's full **262K** by dropping the cache to 4-bit — right up against the edge of the card's memory. That top figure is roughly 196,000 words: a long novel, a year of meeting notes, or a thick stack of contracts, all in one prompt.

Two things worth noting. First, the speed barely moves as the context grows — the model reads a huge prompt about as fast as a small one, because most of its layers use a lighter form of attention that does not balloon with length. Second, the 4-bit cache is the trade-off you pay for the very largest window: less precise memory of what it read, and only a few hundred megabytes of headroom left. For everyday use, the 8-bit / 128K setting is the sweet spot.

## Does squeezing it that hard break it?

That was my worry — a model stored at under two bits per weight sounds like it should be a lobotomised version of itself. It is not. On the checks I ran it stayed sharp: a multi-step arithmetic word problem came out correct, a "write a function to check balanced brackets" request produced clean, idiomatic code with the right answers, and it explained the trade-offs of its own compression method accurately. Nothing garbled, no repetition, no collapse. The compression is impressive precisely because it is so hard to notice.

## The catch

Last month's post ended with two commands and [Ollama](https://ollama.com). This one cannot, and it is only fair to say so. The ternary format is custom — standard tools like Ollama and the usual `llama.cpp` build refuse to load it. You have to build the model author's own version of `llama.cpp` first:

```sh
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j
```

That compiles in a few minutes on a normal machine. Then download the 5.95GB model file and run it:

```sh
./build/bin/llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -fa on -c 65536 \
  -st -p "Summarise the document I am about to paste."
```

To open it up to the full 262K context, add a 4-bit cache: `-ctk q4_0 -ctv q4_0 -c 262144`.

It is a real trade-off: you buy the smaller size, the extra speed, and the huge context with a bit more setup, and a dependency on one team keeping their build going. For a quick try, last month's Ollama route is still the easier door. For a model you plan to lean on — long documents, private data, all day — this one earns the extra step.

## Where this leaves a small business

Five posts, one unchanged computer. The same RTX 3060 has gone from a single still image, to video, to video with sound, to a reasoning model that spilled over the edge of the card — and now to a 27B model that fits entirely on it, runs at a comfortable reading pace, and can swallow a novel's worth of your own documents in a single prompt, without a byte leaving the building.

That last part is the point. A context this large on a private, paid-for-once card means you can hand the model a whole contract, a full support-ticket history, or a year of notes and ask real questions about it — with none of it going to someone else's server, and no per-page cloud bill. The card did not get bigger. We just got much better at using it.

*It's a five-year-old graphics card. What it can do keeps surprising me.*


100a41bc708fc536b7140f337f846e8d7f5c6f80db39818675d03b244d8cbec2

How ternary quantization shrinks a 27B model to 5.95GB so it runs fully on a consumer RTX 3060, roughly four times faster than the 4-bit version, plus a measured look at exactly how much context fits in 12GB of VRAM: 64K by default, 128K or the full 262K with a quantized KV cache.

The fifth in our RTX 3060 series. The same Qwen3.8-27B we ran last month, re-quantized to under 6GB, now runs entirely on the 12GB card, about four times faster, and holds up to 262,000 tokens of context. A whole novel, on a five-year-old gaming card.

Now It Remembers: A 27B Model With a 262K-Token Context on the Same RTX 3060

{/* banner prompt: A playful, polished 3D clay-render in matte sage green, cream and terracotta: a stylized NVIDIA RTX 3060 graphics card holding a large traditional Chinese ink brush, painting a tiny misty mountain landscape onto a small scroll of cream rice paper, a little red seal-stamp prop and an ink stone beside it, soft studio shadows, simple pale background, 16:9. */}

Four posts into this series, the same RTX 3060 has learned to make
[images](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/),
[video](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/),
[video with sound](/blog/2026/08/04-now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/),
and even [reason like a 27B language model](/blog/2026/08/18-now-it-thinks-a-27b-language-model-on-the-same-rtx-3060/).
This time I wanted to point it at something harder to fake: a *specific,
demanding art tradition*. Not "an image in the style of," but **traditional
Chinese ink-wash painting — 水墨 (shui-mo) — on rice paper.**

It's a good test because the style is unforgiving. Real ink painting lives in
what *isn't* painted: the empty rice paper, the 留白 (liú bái) negative space,
the way a single dry-brush stroke suggests a whole pine tree. Get it slightly
wrong and it looks like a greeting card. So: can a five-year-old graphics card,
sitting in a home office, paint a convincing misty-mountain landscape? And how
close does it get to the cloud giants?

## The machine (still the same one)

Same RTX 3060, 12GB of VRAM, 64GB of system RAM, that this whole series is built
around — a card that shipped back in **2021**. Nothing upgraded.

## The test

One prompt, given to every model — a classic 山水 (shan-shui, "mountain-water")
scene:

> *Traditional Chinese ink wash painting on xuan rice paper: misty mountains, a
> lone fisherman's boat, gnarled pines on the cliffs, drifting fog, generous
> negative space, monochrome ink with grey gradients, a small red seal.*

Three models ran **locally on the 3060**. Three ran in the **cloud**, for
comparison.

## The local heroes — painted on the 3060

Here's the headline: **it works, and it works beautifully.** The trick isn't a
giant model — it's a tiny add-on. You take a general open image model already
running on the card and bolt on a **LoRA**: a small (~40–55MB) "style adapter"
trained on Chinese ink paintings. Drop the file in, add a trigger word, and the
model paints in that style.

**SDXL + a Chinese-ink LoRA** gave the most *authentic* result — loose,
expressive, with the faint colour washes and worn look of a scroll that has hung
on a wall for a century:

![A Chinese ink-wash landscape generated locally on an RTX 3060 with SDXL and a Chinese-ink LoRA: misty mountains, a small boat with two figures, pine trees, on aged rice paper](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/sdxl-ink.png)

**FLUX.1-dev + an ink-style LoRA** was the more *refined* look — crisper pines,
deeper atmospheric mist, a flawless composition. It reads as premium modern
"digital ink" rather than a genuine antique, but as an image it's stunning:

![A Chinese ink-wash landscape generated locally on an RTX 3060 with FLUX and the zyd232 ink-style LoRA: sharp misty peaks, pine trees on cliffs, a lone boatman on a still river](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/flux-zyd232.png)

Swap in a *different* ink LoRA and Flux shifts mood — softer, mistier, the peaks
dissolving further into fog — proof that the "look" is a small file you can
change, not a whole new model to download:

![A softer, mistier Chinese ink landscape from FLUX with a second ink-style LoRA (liujiyou), peaks dissolving into fog](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/flux-liujiyou.png)

Each took **about a minute** on the 3060. Both came out of a card that cost less
than a phone, in a home office, for the price of the electricity.

## The honest numbers

<Table data={
    {
        headers: ['', 'Local (RTX 3060)', 'Cloud'],
        rows: [
            ['Setup', 'Forge + a ~50MB LoRA file', 'An API key'],
            ['Time per image', 'About 1 minute', 'Seconds to a minute'],
            ['Cost', 'Electricity', 'A fee per image'],
            ['Privacy', 'Nothing leaves the room', 'Sent to a server'],
            ['Chinese calligraphy', 'Decorative gibberish', 'Legible (gpt-image)'],
            ['Best for', 'Authentic painterly landscapes', 'When you need real text and seals'],
        ]
    }
} />

## The cloud ceiling

To see how far ahead the big players are, I gave the same prompt to three cloud
models. This is where you see both the ceiling — and the one thing local can't
yet touch.

**OpenAI's gpt-image** was the standout of the entire test. Not just
museum-grade brushwork, but **real, legible calligraphy** — it wrote an actual
classical couplet by the poet Du Fu, with a proper signature line and seal
stamps. Look at the characters in the top-left; they're *real writing*, not
decoration:

![A highly authentic Chinese ink landscape by OpenAI gpt-image, with real legible calligraphy, a signature line and red seals](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/openai-gptimage.png)

**Zhipu's GLM (CogView-4)**, a Chinese model, produced a clean, textbook
shan-shui — confident and correct, if a little safe:

![A clean monochrome Chinese ink landscape by Zhipu GLM CogView-4](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/glm-cogview.png)

**Google's Gemini** did something unexpected — it rendered the painting as a
fully *mounted hanging scroll on a wall*, silk border and all, plus two zoomed
detail crops. Impressive, but more a product photo than a painting:

![Google Gemini's take, showing the ink painting as a mounted hanging scroll on a wall with detail crops](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/now-it-paints-chinese-ink-landscapes-on-the-same-rtx-3060/gemini.png)

## What I learned: the writing is the tell

Here's the interesting part. On the *painting* — the mountains, the mist, the
pines, the mood — the local 3060 results genuinely hold their own against the
cloud. Side by side, you'd struggle to pick the home-GPU images out of a lineup.

The gap is the **calligraphy.** Every local model painted beautiful *decorative*
brushstrokes where the inscription should be — squiggles that look like writing
but say nothing. Only gpt-image produced characters you can actually read. If
your painting needs a real poem or a legible seal, that's still a cloud job.
For everything else — the art itself — the card on your desk is enough.

## Where this leaves a small business

If you sell tea, run a restaurant, teach calligraphy, or just want a beautiful,
culturally-grounded header image for a newsletter, you no longer need a stock-art
subscription or a design agency. A modest computer you may already own can paint
you an original ink landscape in about a minute, privately, for the cost of the
power it draws — and you can generate a hundred variations until one is right.

The cloud still wins when the words on the page have to be real. But the *art*?
That quietly moved into the home office, onto a graphics card from 2021.

*Same card. Five years old. Now it paints.*


215c8cbaee653c59092628118433dd6fab0816de09b333db4b8e16dd3d1366b8

Generating traditional Chinese ink-wash painting (shui-mo / guohua) locally on a consumer RTX 3060 with SDXL and FLUX plus ink-style LoRAs, compared against cloud models (OpenAI gpt-image, Google Gemini, Zhipu GLM/CogView): real results, honest limits, and the one thing local models still can't do.

The fifth in our RTX 3060 series. We test whether a five-year-old 12GB graphics card can paint traditional Chinese ink-wash landscapes (shui-mo) on rice paper — locally, with open models — and pit the results against the cloud heavyweights.

Now It Paints: Chinese Ink Landscapes on the Same RTX 3060

{/* banner prompt: A playful, polished 3D clay-render in matte sage green, cream and terracotta: a stylized NVIDIA RTX 3060 graphics card with a softly glowing rounded brain resting on top of it, a few small thought-bubble and gear props floating beside it to suggest reasoning, soft studio shadows, simple background, 16:9. */}

The first three posts in this series were about *making* things — the same RTX 3060
learning to generate [images](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/),
then [video](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/),
then [video with sound](/blog/2026/08/04-now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/).
This one is different. This time the card isn't drawing or filming anything — it's
*thinking*. I put a **27-billion-parameter language model on the same 12GB card**,
and it works.

The headline: **Qwen3.8-27B**, a reasoning model released only a few days ago, runs
locally on a consumer RTX 3060 that shipped back in **2021**. It answers questions, writes code,
reasons step by step — and it can even look at a picture and tell you what's in it.
The kind of model people assume needs a data centre, or at least a $2,000 graphics
card, runs on hardware a small business already owns.

## The machine (still the same one)

Same box as every post in this series: an RTX 3060 with **12GB of VRAM**, 64GB of
system RAM, Debian. Nothing upgraded. The card is the constant.

## Why this is surprising

A 27B model is *big*. The rule of thumb people quote is that you need a 24GB card
(an RTX 4090 or better) to run one at all — and its full-precision form is over
50GB, firmly data-centre territory. So "27B on 12GB" sounds like it shouldn't be
possible.

It is, because of two things working together: **quantization** and **offloading.**

## Fitting an 18GB model into 12GB

Quantization compresses the model's numbers to 4 bits each, shrinking it from 50GB+
down to about **18GB** with very little quality lost. That still doesn't fit in
12GB — but it doesn't have to. The card holds a bit more than half the model
(**about 57%**), and the rest lives in the 64GB of system RAM, streamed onto the
GPU as it's needed. The GPU does the heavy math; the spare layers wait in RAM.

The remarkable part is how *easy* it was. The video-with-sound model in the last
post took patches and careful memory tuning. This time it was genuinely two
commands — download, and run. The tool (Ollama) works out the GPU/RAM split
automatically.

## The honest numbers

<Table data={
    {
        headers: ['', 'On the RTX 3060'],
        rows: [
            ['Model', 'Qwen3.8-27B (released this month)'],
            ['Size on disk', '~18GB (4-bit)'],
            ['Runs on GPU', '~57% on the card, ~43% streamed from RAM'],
            ['VRAM used', '11 of 12 GB'],
            ['Speed', '~4.7 words-per-second (reads about as fast as you do)'],
            ['Can it see images?', 'Yes — it is multimodal'],
            ['Best for', 'Questions, coding help, explanations, one-image analysis'],
        ]
    }
} />

Four-ish words a second isn't fast — you watch the answer type itself out, roughly
at reading speed. This is not the tool for churning through a thousand requests an
hour. It's the tool for asking one good question and getting a thoughtful answer,
privately, on your own machine, for the cost of the electricity.

## It reasons — out loud

Qwen3.8 is a *reasoning* model: before it answers, it thinks. Ask it a coding
question and you can watch it work the problem — considering approaches, catching
its own mistakes, checking its logic — then give a clean final answer. Ask it to
find the longest palindrome in a string, and it doesn't just spit out code; it
reasons about edge cases and complexity first. That visible thinking is a big part
of why these newer models are so much more useful than the ones from a year ago.

## And it sees

Here's the part that closes the loop on this whole series. The model is multimodal,
so I showed it one of the AI-generated animal photos from our image experiments and
asked: *is this a real photograph, or AI-generated?*

It nailed it. It described the scene correctly — a brown bear in a rushing river,
autumn forest blurred behind it — and then, unprompted, listed exactly *why* it
looked artificial: fur that was "suspiciously smooth and painted," the
"characteristic milky AI bokeh" in the background, "melted" rocks, a glassy eye
lacking real reflections. A local model, on a 12GB card, giving a genuinely sharp
critique of a locally-generated image. The tools in this series can now check each
other's work.

## Try it yourself

If you have an RTX 3060 (or any 12GB card) and [Ollama](https://ollama.com)
installed, this is the whole thing:

```bash
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Q4_K_M
```

The first run downloads the model; after that it's instant to start. Ask it
anything. To feed it an image, just put a file path in your message.

## Where this leaves a small business

Four posts, one unchanged computer. The same RTX 3060 went from a single 512px
still, to short video, to video with a soundtrack — and now to a reasoning model
that can answer questions, help write code, and read an image, all without a single
byte leaving the building.

That last point matters more than the speed. Running the model *locally* means your
questions, your customer data, your documents never touch someone else's server.
No per-message cloud bill, no data leaving your office, no dependency on a
subscription that can change its terms next quarter. For a lot of small businesses,
"good enough, private, and paid-for-once" beats "fastest" every time.

It isn't a data centre. It doesn't need to be.

*It's a five-year-old graphics card. What it can do keeps surprising me.*


948ef2e0820c8a951e9aa975e9f905c44c461234611a88494f1617dd4622ea0a

Running Qwen3.8-27B, a brand-new 27B multimodal reasoning model, locally on a consumer RTX 3060 with only 12GB of VRAM: how an 18GB model fits, honest tokens-per-second, what it can and can't do, and the one command to try it yourself with Ollama.

The fourth in our RTX 3060 series. Qwen3.8-27B — a 27-billion-parameter reasoning model released this month — runs on the same 12GB card. Not in the cloud, not on a $2,000 GPU. One command, and it thinks, codes, and even sees.

Tag: Rtx 3060

Tag: Rtx 3060

Now It Remembers: A 27B Model With a 262K-Token Context on the Same RTX 3060

Now It Paints: Chinese Ink Landscapes on the Same RTX 3060

Now It Thinks: A 27B Language Model on the Same RTX 3060